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Safe separation of aircraft is a primary objective of any air traffic control system. An 
accelerated Monte Carlo approach was developed to assess the level of safety provided by a 
proposed next-generation air traffic control system. It combines features of fault tree and 
standard Monte Carlo methods. It runs more than one order of magnitude faster than the 
standard Monte Carlo method while providing risk estimates that only differ by about 10%. 
It also preserves component-level model fidelity that is difficult to maintain using the 
standard fault tree method. This balance of speed and fidelity allows sensitivity analysis to 
be completed in days instead of weeks or months with the standard Monte Carlo method. 
Results indicate that risk estimates are sensitive to transponder, pilot visual avoidance, and 
conflict detection failure probabilities. 


I. Introduction 

A IR traffic demand is expected to increase substantially over the next 20 years, but controller workload limits 
capacity. 1 It is expected that higher levels of automation for separation assurance (SA) will be required, and 
NASA is investigating both airborne and ground-based automation concepts to handle future demand growth. 2 "' A 
crucial step toward deployment is to assess the level of safety provided by candidate SA systems. The subject of this 
analysis is the Advanced Airspace Concept (AAC), a proposed ground-based next-generation air traffic control 
system. 3 ' 5 

Two AAC risk analysis studies have been conducted recently. The first used a fault tree approach to study four 
fault types: (1) nominal conditions, (2) information fault non-conformance, (3) control fault non-conformance, and 
(4) service interruption. 6 Results of this study suggest AAC could achieve the safety levels expected in 20-30 years 
given the deployment of appropriately designed safety features. Although risk estimates can be computed quickly 
using the fault tree method, it does not facilitate component-level model fidelity because the number of conditional 
probabilities and failure modes grows exponentially. 

The second AAC risk analysis study used a standard Monte Carlo method with fault tree style logic built into the 
simulation. 7 In this study, each safety layer of AAC was decomposed into its hardware and software subcomponents. 
Critical components shared between different AAC safety layers (e.g., on-board transponder and conflict resolution 
maneuver readers) and common failure modes were identified. The estimated safety level of AAC in this study met 
the desired target for an automated separation assurance function. The standard Monte Carlo approach had a higher 
level of model fidelity compared to the fault tree method. However, the tradeoff was a lengthy simulation runtime of 
around 1 6 hours. As such, it is impractical to conduct a thorough sensitivity analysis using the standard Monte Carlo 
method. 

The current study develops an accelerated Monte Carlo approach that combines features of both fault tree and 
standard Monte Carlo methods. The first step in this accelerated method applies the structure of the fault tree method 
to divide AAC into a set of six subfunctions and identify five unique failure modes. Then, the probability of each 
failure mode is estimated through separate smaller Monte Carlo simulations and summed up into overall probability 
estimates of critical events such as near mid-air collisions (NMAC) and mid-air collisions. Since AAC is a proposed 
next-generation system with components that are not deployed in the National Airspace System (NAS), sensitivity 
analysis is important to identify safety-critical components that require further study. The reductions in simulation 
runtime achieved using the accelerated Monte Carlo approach facilitate such sensitivity analysis. 

Section II provides additional detail about AAC and the components that comprise each of its subsystems. 
Section III focuses on the accelerated Monte Carlo approach. Section IV compares the simulation runtime and 
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NMAC and mid-air collision estimates for the standard and accelerated Monte Carlo approaches. Section V analyzes 
the sensitivity of NMAC and mid-air collision estimates to component failure probabilities using the accelerated 
Monte Carlo approach. Section VI discusses the results of the experiment. Section VII presents the conclusions. 

II. Background 

The Advanced Airspace Concept (AAC) is a proposed ground-based separation assurance system that monitors 
and maintains safe separation between aircraft automatically. Trajectory changes initiated by pilots and/or the 
ground system are communicated via air-ground data link. AAC includes two safety levels, Autoresolver (AR) and 
Tactical Separation- Assured Flight Environment (TSAFE). In addition, two safety layers already present in today’s 
system are retained: Traffic alert Collision Avoidance System (TCAS) and visual avoidance by pilots. While AAC is 
responsible for resolving routine conflicts, controllers will still handle emergencies and special pilot requests. Figure 
1 shows AAC’s conflict detection and resolution elements and their respective action time ranges. 



Figure 1: AAC conflict detection and resolution timeline 


The Autoresolver (AR) is the workhorse of AAC. It looks ahead three to twenty minutes to detect and prevent 
losses of separation (LoS), defined as separation of less than five nautical miles (nmi) of horizontally and 1000 feet 
(ft) vertically. Its purpose is to handle nearly all conflicts that arise. It receives a continual feed of positional data for 
flights via radar and Global Positioning System (GPS) units onboard most aircraft, from which it generates 
trajectory predictions. Its conflict detection function probes predicted trajectories two-at-a-time for all pairs of 
flights in the region of airspace for which it is responsible. Then, its conflict resolution algorithm searches for 
trajectories predicted to be conflict-free for at least 20 minutes in the future. Fast-time simulations of AR indicate it 
can handle the full breadth and variety of conflict situations that occur in enroute airspace including climbs and 
descents to arrival fixes at up to three times current traffic levels/' 9 

TSAFE is the primary backup system to AR and is also implemented on the ground. Its purpose is to detect and 
resolve conflicts in the shorter tactical time horizon of zero to three minutes prior to LoS that were not detected or 
were not resolved by AR at an earlier time to LoS. 10 ’ 12 As such, it is designed to be simpler than AR and places 
higher priority on safety over efficiency. TSAFE is an autonomous system on the ground that runs in parallel to AR 
and relies on the same data. As such, it has more accurate resolutions compared to TCAS (described below). 
However, it is similar to TCAS in using on-board voice synthesizers to issue heading or altitude changes to pilots. 

TCAS is the safety net beneath AR and TSAFE. It is an airborne collision avoidance system that is independent 
of both onboard navigation equipment and ground-based air traffic control systems including AR and TSAFE. 13 
Each TCAS-equipped aircraft interrogates nearby aircraft about their respective current altitudes and also responds 
with its own altitude when interrogated. Each individual aircraft’s TCAS system repeats the interrogation process 
continually to determine the range of nearby aircraft. An on-board computer analyzes the data to generate collision 
avoidance maneuvers. Resolution advisories (climb or descend) are issued via cockpit display and synthesized voice 
commands. 

Visual detection and avoidance by the pilot constitute the final safety layer. This is the last resort if the other 
automated separation assurance systems are all unable to detect and resolve an impending collision 
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III. Accelerated Monte Carlo Simulation Approach 

This section describes the accelerated Monte Carlo simulation method for a safety analysis of AAC. It combines 
features of both fault tree and standard Monte Carlo methods. The accelerated Monte Carlo approach first applies 
the structure of the fault tree method to divide AAC into six distinct subfunctions and identify five unique failure 
modes. Then, the probability of each failure mode is rewritten as a product of several conditional probabilities using 
analogues of Bayes’ Theorem. After that, each failure mode probability is estimated using Monte Carlo simulations 
of reduced complexity. 

A. Simulation Assumptions and Simplifications 

The simulations conducted in this study are derived from those run in a prior safety analysis of AAC. 7 As such, 
all assumptions and simplifications made in the previous study are also present here. They must be examined more 
closely because they can affect the estimates ofNMAC (defined as separation of less than 500 ft vertically and 100 
ft horizontally 6 ) and collision risk. An abridged list of the most important ones is given below: 

• In the absence of any SA systems, NMACs are independent, identically distributed events. 

• AAC component failures are independent and identically distributed. 

• Average flight time is two hours. 

• Flights are locatable either via Automatic Dependent Surveillance-Broadcast (ADS-B) or Secondary 
Surveillance Radar (SSR), but primary radar is not used. 

• Maximum look-ahead time for conflict detection and resolution is eight minutes. 

• AR and TSAFE perform conflict detection and resolution and issue maneuvers in 30-second intervals. 

• Component failures are not relevant to the simulation unless they occur within eight minutes prior to 
NMAC. Otherwise, it is assumed that failure monitors will identify failures and perform mitigating actions. 

• Component failures are only in effect if they occur between eight minutes prior to NMAC and the current 
time step in the simulation. 

• Probability of detecting a conflict only depends on look-ahead time (e.g., phase of flight is not a factor). 

• If a conflict-free resolution is found and successfully communicated, it is executed correctly, resolves the 
conflict in all traffic situations, and there is no conflict between the same flight pair in the future. 

• Flights are equipped with ADS-B and TCAS Version II. 

B. Subfunction Identification 

The first step in the standard Monte Carlo risk analysis of AAC was to decompose each safety layer in Figure 1 
into its respective hardware, software, and functional subcomponents. 7 (See the Appendix for fault tree diagrams 
and descriptions and failure probabilities of these components.) In the accelerated Monte Carlo approach, the first 
step is to gather them into subfunctions. Simulation runtime is minimized by minimizing the number of subfunctions 
necessary to capture all failure modes possible in the standard Monte Carlo method. Analysis indicated that this 
could be achieved with six subfunctions: 1) flight locatability, 2) AR conflict detection, 3) AR conflict resolution 
and communication, 4) TSAFE conflict detection, 5) TSAFE resolution and communication, and 6) TCAS and pilot 
visual avoidance. (See Table 1 for subfunction abbreviations.) 


Subfunction 

Abbreviation 

Flight locatability (via ADS-B or SSR) 

L 

AR conflict detection 

s d 

AR conflict resolution and communication 

s r 

TSAFE conflict detection 

T d 

TSAFE conflict resolution and communication 

T r 

TCAS and pilot visual avoidance 

A 


Table 1: AAC subfunctions 


C. Failure Mode Identification 

Using the fault tree method, five unique failure modes were identified for these six subfunctions. All involve 
TCAS not functioning on at least one flight and: 

• At least one flight not locatable via either ADS-B or SSR, or 
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• Both flights are locatable and the conflict is detected by AR and TSAFE, but both AR and TSAFE cannot 
find or communicate a conflict-free resolution, or 

• Both flights are locatable and detected by AR, but a conflict-free AR resolution could not be found or 
communicated, and the conflict was not detected by TSAFE, or 

• Both flights are locatable but the conflict is only detected by TSAFE and a conflict-free TSAFE resolution 
was not found or communicated, or 

• Both flights are locatable, but AR and TSAFE do not detect the conflict. 

These five failure modes are represented in Table 2. Green boxes are AAC subfunctions that are operational in 
the given failure mode. By contrast, red boxes denote failed subfunctions. Red arrows indicate how failure of one 
subfunction (arrow’s origin) also causes another subfunction downstream to fail (arrow’s destination). For instance, 
failure mode 5 consists of cases where both flights are locatable. However, since AR does not detect the conflict in 
this failure mode, no strategic AR resolution is found or communicated. Similarly, since TSAFE does not detect the 
conflict, no tactical TSAFE resolution is issued. Lastly, as in all failure modes, TCAS and pilot visual avoidance do 
not detect and avoid the NMAC, which also has a chance of ending up as a mid-air collision. 



Table 2: Component states in failure modes (accelerated Monte Carlo) 


Table 3 contains two sets of equations derived from these failure modes that are used to estimate the rate of 
NMACs and collisions, respectively. The only difference between them is a “C” term in the latter set of equations 
for the event of collision. 


Failure mode 
number 

Probability of NMAC 

Probability of collision 

1 

P(N0 = P(L’, A’) 

P(C() = P(C, L’, A’) 

2 

P(N 2 ) = P(L, S d , S r \ T d , T r \ A’) 

P(C 2 ) = P(C, L, S d , S r ’, T d , T r \ A’) 

3 

P(N 3 ) = P(L, S d , S r ’,T d ’,A’) 

P(C 3 ) = P(C, L, S d , S r ’, T d ’, A’) 

4 

P(N 4 ) = P(L, S d ’, T d , T r \ A’) 

P(C 4 ) = P(C, L, Sd’, Td, T r ’, A’) 

5 

P(N 5 ) = P(L, S d ’,T d ’,A’) 

P(C 5 ) = P(C, L, S d ’, T d ’, A’) 

Total 

P(N) = P(Nj) + P(N 2 ) + P(N 3 ) + P(N 4 ) + P(N 5 ) 

P(C) = P(Q) + P(C 2 ) + P(C 3 ) + P(C 4 ) + P(C 5 ) 


Table 3: Failure mode equations 


D. Failure Mode Decomposition 

Each failure mode was rewritten using analogues of Bayes’ Theorem as shown in Tables 4-5 for NMAC and 
collision, respectively. Each tenn in these equations is simulated from left to right using the results of the preceding 
simulation as a set of possible initial conditions for the next one. This eliminates the need to step through billions of 
simulation trials one at a time as in the standard Monte Carlo method. In this respect, the accelerated Monte Carlo 
method is similar to recent studies that used rare event Monte Carlo simulation to estimate the collision risk of an 
uncoordinated airborne self-separation concept. 14 ' 15 

Consider failure mode 1. The first step in estimating its probability is to simulate the likelihood that at least one 
flight is not locatable via ADS-B or SSR in a Monte Carlo simulation. The result is an estimated value for P(L’) and 
a set of “non-locatable” cases. 

The next step is to estimate the probability that TCAS and pilots fail to avoid the NMAC given that at least one 
flight is not locatable via ADS-B or SSR. At the start of each Monte Carlo simulation trial, one “non-locatable” case 
identified in the first step is selected at random and used as the initial conditions. The end result of the simulation 
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trial is an estimated value for P(A’|L’). The probability of NMAC in failure mode 1 is then computed as the product 
of P(L’) and P(A’|L’) as shown in Table 4. The probability of collision (see Table 5) can be estimated using the set 
of “non-locatable and TCAS and pilot visual avoidance failure” cases as initial conditions in a similar way. 


Failure 

mode 

Failure mode probability of NMAC 

N! 

P(Ni) = P(L’)- P(A’|L’) 

n 2 

P(N 2 ) = P(L) • P(S d , S r ’|L) • P(T d , T r ’|S r \ S d , L) P(A’|T r \ T d , S r \ S d , L) 

n 3 

P(N 3 ) = P(L)- P(S d , S r ’|L )• P(T d ’|S r ’, S d , L)- P(A’|T d \ S r \ S d , L) 

n 4 

P(N 4 ) = P(L). P(S d ’|L)- P(T d , T r ’|S d \ L) • P(A’|T r ’, T d , S d \ L) 

n 5 

P(Nj) = P(L). P(S d ’|L)- P(T d ’|S d ’, L) P(A’|T d ’, S d \ L) 

Total 

P(N) = P(Nj) + P(N 2 ) + P(N 3 ) + P(N 4 ) + P(N 5 ) 


Table 4: Bayes’ theorem decomposition of failure modes for NMAC 


Failure 

mode 

Failure mode probability of collision 

Q 

P(C() = P(L’) P ( A ’ | L ’ ) P(C|A’, L’) 

c 2 

P(C 2 ) = P(L)- P(S d , S r ’|L) P(T d , T r ’|S r \ S d , L) P(A’|T r \ T d , S r \ S d , L)- P(C|A\ T r \ T d , S r \ S d , L) 

c 3 

P(C 3 ) = P(L)- P(S d , S r ’|L) ■ P(T d ’|S r \ S d , L) P(A’|T d ’, S r ’, S d , L)- P(C|A’, T d ’, S r ’, S d , L) 

c 4 

P(C 4 ) = P(L)- P(S d ’|L)- P(T d , T r ’|S d ’, L)- P(A’|T r ’, T d , S d ’, L)- P(C|A’, T r ’, T d , S d ’, L) 

c 5 

P(C 5 ) = P(L)- P(S d ’|L)- P(T d ’|S d ’, L)- P(A’|T d ’, S d ’, L)- P(C|A\ T d \ S d ’, L) 

Total 

P(C) = P(C0 + P(C 2 ) + P(C 3 ) + P(C 4 ) + P(C 5 ) 


Table 5: Bayes’ theorem decomposition of failure modes for collision 


Note that some AAC subfunctions are grouped together and simulated at the same time. For instance, probability 
estimates for AR detection and resolution come from a single simulation; this was also the case for the probability 
estimates for TSAFE detection and resolution. This is because both conflict detection and resolution are performed 
in sequence at regular intervals in both AR and TSAFE. 

Note that the number of simulations needed is less than the number of factors shown in Tables 4 and 5 since 
several factors are shared by multiple failure modes (e.g., P(L)). As such, only one simulation run for that factor is 
necessary. In some cases, multiple factors can be estimated in the same simulation. For instance, in simulation trials 
to estimate P(L), only two possible outcomes are possible: 1) both flights are locatable, or 2) at least one flight is not 
locatable. As a result, estimates of both P(L) and its complement P(L’) = 1 - P(L) come from the same simulation. It 
turns out that the five failure mode probabilities can be estimated with a total of just nine Monte Carlo simulations. 

IV. Simulation Results 

This section compares simulation results for the accelerated and standard Monte Carlo methods. All simulations 
were performed using Java on the same machine with a dual-core processor and clock speed of 3.06 GFlz and 8 GB 
of memory. Simulation parameters including component failure probabilities can be found in the Appendix. 

The accelerated Monte Carlo approach is designed to reduce simulation runtime while maintaining component- 
level fidelity. As such, the simulation runtimes and estimates of NMAC and collision risk are compared for both the 
standard and the accelerated Monte Carlo methods. 

A. Runtime Comparison 

In the case of the standard Monte Carlo simulation with ten billion trials, total runtime was 15.7 hours. Although 
performing fewer simulation trials can reduce runtime, it will also result in unacceptably small sample sizes for rare 
safety-critical events such as NMACs and collisions. 

The accelerated Monte Carlo approach ran in just 4% of the time needed by the standard Monte Carlo method. A 
breakdown of the nine simpler Monte Carlo simulations performed can be found in the last column of Table 6. The 
first column indicates the AAC component that was simulated. The next six columns indicate the state of the other 
AAC sub functions in the set of possible initial conditions. As in Table 2, green boxes indicate sub functions that are 
operational, and red boxes denote those that have failed. White boxes are ones that have not yet been simulated. 
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Simulated 

Initial conditions 

Runtime (min) 

component 

L 

s d 

s r 

T„ 

T r 

A 


Aircraft Locatability 







2.8 

Strategic (AR) 







15.1 

Tactical (TSAFE) 







9.5 

Tactical (TSAFE) 







11.2 

TCAS/Pilot/C 







0.0 

TCAS/Pilot/C 







0.0 

TCAS/Pilot/C 







0.0 

TCAS/Pilot/C 







0.0 

TCAS/Pilot/C 







0.0 

Total 


38.6 


Table 6: Breakdown of simulation runtimes (accelerated Monte Carlo) 


B. Risk Estimate Comparison 

The reductions in runtime that could be achieved in the accelerated Monte Carlo approach are not of value unless 
its NMAC and collision risk estimates are similar to those using the standard Monte Carlo method. The fraction of 
simulation trials that experienced NMAC in ten billion trials in the standard Monte Carlo approach was 2.6 ■ 10" . 

By comparison, the accelerated Monte Carlo approach had an NMAC estimate of 2.9 • 1 0 8 , which is a difference of 
about 1 1%. See Table 7 for detailed failure mode estimates. 


Failure Mode 

Probability description 

Probability 

Nr 

P(N0 = P(L\ A’) 

2.7 • 10" 

n 2 

P(N 2 ) = P(L, S d , S r \ T d , T r \ A’) 

3.9 • 10 14 

n 3 

P(N 3 ) = P(L, S d , S r \ T d \ A’) 

6.6 10 12 

n 4 

P(N 4 ) = P(L, S d \T d , T r ’,A’) 

8.5 - 10 12 

n 5 

P(N S ) = P(L, S d \ T d \ A’) 

1.7 • 10 9 

Total 

P(N) = P(N0 + P(N 2 ) + P(N 3 ) + P(N 4 ) + P(N 5 ) 

2.9 10" 


Table 7: NMAC probability estimates by failure mode (accelerated Monte Carlo) 

The collision probability estimates were also similar (see Table 8). The standard Monte Carlo simulation with 
ten billion trials had a collision probability estimate of 8.9 ■ 10" . By comparison, the estimated collision probability 
using the accelerated Monte Carlo approach was 9.7 • 1 0 9 , which is a difference of about 9%. 


Failure Mode 

Probability description 

Probability 

Ci 

P(C 1 ) = P(C,L’,A’) 

9.0 10 9 

c 2 

P(C 2 ) = P(C, L, Sd, S/, T d , T r ’, A’) 

1.4 • 10 14 

C 3 

P(C 3 ) = P(C, L, Sd, S r ’, T d ’, A’) 

2.5 ■ 10" 2 

c 4 

P(C 4 ) = P(C, L, S d ’, T d , T r \ A’) 

3.3 1 0 12 

C 5 

P(C 5 )= P(C, L, Sd’, Td’, A’) 

6.0-10"° 

Total 

P(C) = P(C0 + P(C 2 ) + P(C 3 ) + P(C 4 ) + P(C 5 ) 

9.7-10" 


Table 8: Collision probability estimates by failure mode (accelerated Monte Carlo) 

Additional Monte Carlo simulations were needed to estimate the time between NMACs and mid-air collisions 
based on the probability estimates in Tables 7 and 8. This took another 24 minutes, which resulted in an overall 
simulation runtime of 63 minutes. This is equal to about 7% of the runtime required by the standard Monte Carlo 
method. 

Results from these simulations are presented in Table 9. The estimated mean time between NMACs and 
collisions using the accelerated Monte Carlo approach is about 10% less than estimates using the standard Monte 
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Carlo method. These results were as expected because the NMAC and collision probability estimates also differed 
by about 10%. 



NMAC 

Collision 


Accelerated 

Standard 

Accelerated 

Standard 

Number of trials 
(out of 10 billion) 

291 

262 

103 

89 

Mean time between 
events 

87 years 

96 years 

246 years 

281 years 

Sample standard 
deviation of time 
between events 

84 years 

96 years 

243 years 

244 years 


Table 9: Estimates of time between NMACs and mid-air collisions 


V. Sensitivity Analysis 

The similarity of NMAC and collision risk estimates using the standard and accelerated Monte Carlo approaches 
indicates that the latter can be used for sensitivity analysis. This was previously identified as a topic for future AAC 
risk analysis research. 7 This can be useful for identifying safety-critical components of AAC that require further 
study, especially those that are not deployed in today’s NAS. The results of a limited sensitivity analysis using the 
accelerated Monte Carlo approach are presented below. 

Sensitivity analysis was first performed on components that are common to multiple failure modes: 1) mode S 
transponder, 2) resolution reader, and 3) speaker failure. Failure probabilities were changed by one and two orders 
of magnitude. The estimated risk of NMACs and collisions did not change much for the latter two components; 
NMAC estimates remained between 70 and 100 years, and collision estimates stayed between 200 and 300 years. By 
comparison, they were noticeably sensitive to changes in transponder failure as expected (see Table 10). This is 
because its nominal failure probability is one of the highest in the model, and it is used for locatability via ADS-B 
and SSR as well as by TSAFE and TCAS. This indicates that additional research to get more accurate transponder 
failure probability is needed. 


Failure probability 

Time between NMACs (yrs) 

Time between collisions (yrs) 

Runtime (min) 


Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 


Nominal P(RT) 

87 

84 

246 

243 

62.6 

Nominal P(RT)- 10 

53 

55 

157 

161 

58.0 

Nominal P(RT)- 100 

3 

3 

10 

11 

55.6 


Table 10: Mode S transponder sensitivity analysis results 


Sensitivity analysis was also performed on the failure probability of pilot visual avoidance, and the probability of 
mid-air collision given NMAC. This is because they directly affect the proportion of conflicts not resolved by AR, 
TSAFE, or TCAS that become NMACs and mid-air collisions. As expected, the risk estimates for both NMAC and 
mid-air collision events were found to be sensitive (see Tables 1 1 and 12). As such, the failure probability of pilot 
visual avoidance and the probability of mid-air collision given NMAC also require additional investigation. 


Failure probability 

Time between NMACs (yrs) 

Time between collisions (yrs) 

Runtime (min) 


Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 


Nominal P(VA) = 0.3 

87 

84 

246 

243 

62.6 

P(VA) = 0.4 

71 

66 

233 

208 

61.5 

P(VA) = 0.5 

46 

45 

141 

143 

59.3 


Table 11: Pilot visual avoidance sensitivity analysis results 
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Failure probability 

Time between NMACs (yrs) 

Time between collisions (yrs) 

Runtime (min) 


Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 


Nominal P(CNMAC) 
= 0.28 

87 

84 

246 

243 

62.6 

P(CNMAC) = 0.4 

124 

103 

174 

155 

60.2 

P(CNMAC) = 0.5 

155 

187 

143 

176 

63.2 


Table 12: Mid-air collision given NMAC sensitivity analysis results 


Lastly, the sensitivity of NMAC and collision estimates to both strategic and tactical conflict detection failure 
probability was also analyzed. This is important because of a modeling assumption that the probability of detecting a 
conflict only depends on look-ahead time and no other factor including phase of flight. 7 As shown in Table 13, 
changes in NMAC and mid-air collision risk estimates are apparent for just a 20% increase in detection failure 
probability. This points to the need for further study to improve estimates of AAC conflict detection probabilities. 


Failure probability 

Time between NMACs (yrs) 

Time between collisions (yrs) 

Runtime (min) 


Mean 

Standard 

Deviation 

Mean 

Standard 

Deviation 


Nominal P(ACD), 
Nominal P(TCD) 

87 

84 

246 

243 

62.6 

Nominal P(ACD)- 1.1, 
Nominal P(TCD)- 1.1 

72 

71 

227 

209 

64.0 

Nominal P(ACD) • 1 .2, 
Nominal P(TCD)- 1.2 

45 

43 

101 

95 

67.1 


Table 13: Conflict detection sensitivity analysis results 


VI. Discussion 

A. Sources of Simulation Runtime Reductions in the Accelerated Monte Carlo Approach 

One main reason for the reduction in simulation runtime with the accelerated Monte Carlo approach is that fewer 
simulation trials were required. As discussed in Section III, a total of nine Monte Carlo simulations were needed to 
compute NMAC and collision risk estimates. In some cases, it took up to around 10 8 9 trials to get at least 100 cases 
of subfunction failures in a simulation. Recall that these were used as initial conditions in subsequent simulations. 
However, in most cases, the number of trials needed was several orders of magnitude less. As a result, NMAC and 
mid-air collision risk estimates could be made with less than half of the 10 10 simulation trials needed in the standard 
Monte Carlo method. 

B. Additional Sensitivity Analysis 

A limited sensitivity analysis was conducted in this paper as a demonstration of an application of the accelerated 
Monte Carlo approach. It focused on components of AAC that were identified based on fault tree analysis, failure 
probability magnitude, and intuition. A more thorough sensitivity analysis of all the modeled components of AAC 
(see Appendix) is a subject for future research. 

C. Confidence Intervals 

The reduction in simulation runtime with the accelerated Monte Carlo approach also facilitates the construction 
of confidence intervals for NMAC and collision risk estimates in AAC. However, it may be necessary to utilize 
distributed computing with multiple machines because this can require thousands of simulations. This may be an 
area of additional work in the future. 


VII. Conclusions 

An accelerated Monte Carlo approach that combines features of fault tree and standard Monte Carlo methods 
was developed to assess the level of safety provided by the Advanced Airspace Concept. Each simulation using the 
accelerated approach ran in about one hour while providing risk estimates that only differ with those of the standard 
Monte Carlo method by about 10%. This is more than one order of magnitude faster than the 16 hours required in a 
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standard Monte Carlo simulation. It also preserves component-level model fidelity that is difficult to maintain using 
the standard fault tree method. 

Since AAC is a proposed next-generation system with components that are not deployed in the NAS, sensitivity 
analysis is necessary to identify component failure probabilities that require additional research. The speed and 
model fidelity of the accelerated Monte Carlo approach facilitates such sensitivity analysis that can be completed in 
days instead of weeks or months with the standard Monte Carlo method. Results indicate that risk estimates of near 
mid-air collisions and mid-air collisions are sensitive to transponder, pilot visual avoidance, and conflict detection 
failure probabilities. Further investigation to obtain more accurate failure probabilities for these three components is 
needed to generate more accurate estimates of the level of safety provided by AAC. 

Appendix 

One of the primary features of the standard Monte Carlo method is its flexibility for component-level simulation. 
This section describes the individual components of each AAC subfunction and how they interact with one another. 

A. Aircraft Locatability 

Flights are locatable if they can be tracked via ADS-B or SSR. A flight is locatable via ADS-B if the following 
three things are true: 1) it receives position data via GPS, 2) it broadcasts data to ground station via data link, and 3) 
the ground station successfully sends the flight’s data to Host computers. Likewise, a flight is locatable via SSR if 
the following are true: 1) it receives a signal from ground station to send its data, 2) it uses its transponder to send 
data to the ground, and 3) its data is sent from ground station to Host computers successfully. If one or both flights 
are not locatable, then AR and TSAFE cannot handle the conflict. It should be noted that SSR does provide a “skin 
paint” return in the event of transponder failure that could also be used by air traffic controllers to detect and resolve 
conflicts, but this feature is not modeled here. 
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Figure Al: Aircraft 1 locatability 


Component 

Description 

Failure probability 

AG1 

ADS-B: AC1 gets position via GPS 

0.0005 (ref. |T71) 

AB1 

ADS-B: AC1 broadcasts data to ground station 

0.0000097 (ref. [161) 

ADI 

ADS-B Ground station sends ACl’s data to Host 

0.00002 (ref. [171) 

RSI 

Radar: Ground signals AC1 to send data 

0.00682 (ref. [161) 

RT1 

Radar: Transponder sends data to ground 

N/A (RT1=AB1) 

RD1 

Radar: Data Set from ground station to Host comps 

N/A (RD1=AD1) 


Table Al: Locatability components and failure probabilities for an aircraft (AC1) 


B. AR Functions 

AR is not operational unless both aircraft are locatable via SSR or ADS-B as described in the prior section. If 
there is track data for both flights, AR resolves the conflict only if flightplan-based trajectories can be computed for 


9 

American Institute of Aeronautics and Astronautics 


both flights (FPT1 and FPT2), strategic conflict detection and resolution (ACD and ACR) are successful, and the 
resolution trajectory can be communicated by voice or datalink. 



Jl Jl Jl Jl jL jL 

VDL21 RR1 FMS1 VDL22 RR2 FMS2 

Figure 2: AR detection, resolution, and communication 


Component 

Description 

Failure probability 

FPT1 

Generate 4-D flight plan trajectory (AC1) 

0.000001 (assumed) 

FPT2 

Generate 4-D flight plan trajectory (AC2) 

0.000001 (assumed) 

ACD 

Auto-Resolver: Conflict detection module 

Varies (see Appendix, sec. H of ref. [7]) 

ACR 

Auto-Resolver: Conflict resolution module 

0.000001 (ref. [61) 

VDL21 

VDL2: Resolution upload via data link (AC1) 

0.00004 (ref. [181) 

VDL22 

VDL2: Resolution upload via data link (AC2) 

0.00004 (ref. [181) 

VC1 

ACl’s data sent to Flost via voice communication 

0.00055 (ref. [161) 

VC2 

AC2’s data sent to Host via voice communication 

0.00055 (ref. [161) 

RR1 

Onboard resolution reader (AC1) 

0.000001 (ref. [181) 

RR2 

Onboard resolution reader (AC2) 

0.000001 (ref. [181) 

FMS1 

Onboard resolution trajectory generator (AC1) 

0.000097 (proxy from ref. [16]) 

FMS2 

Onboard resolution trajectory generator (AC2) 

0.000097 (proxy from ref. [16]) 


Table A2: AR components and failure probabilities 


C. TSAFE Functions 

If both flights are locatable but AR fails to resolve the conflict, then TSAFE becomes responsible for handling it. 
The TSAFE fault tree is similar to the AR one in that it only resolves the conflict if dead-reckoning trajectories can 
be calculated for both flights (DRT1 and DRT2), tactical conflict detection and resolution (TCD and TCR) are 
successful, and the resolution trajectory can be communicated via Mode S to an on-board resolution reader that 
translates the maneuver and produces an aural command for the pilot to execute. 
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Figure A3: TSAFE detection, resolution, and communication 


Component 

Description 

Failure probability 

DRT1 

Generate 4-D dead reckoning trajectory (AC1) 

0.000001 (assumed) 

DRT2 

Generate 4-D dead reckoning trajectory (AC2) 

0.000001 (assumed) 

MSI 

Mode S: Resolution upload via data link (AC1) 

N/A (MS1=AB1) 

MS2 

Mode S: Resolution upload via data link (AC2) 

N/A (MS2=AB2) 

TCD 

TSAFE: Conflict detection module 

Varies (see Appendix, sec. H of ref. [7]) 

TCR 

TSAFE: Conflict resolution module 

0.000001 (assumed) 

RR1 

Onboard resolution reader (AC1) 

N/A (Same as RR1 in Table A2) 

RR2 

Onboard resolution reader (AC2) 

N/A (Same as RR2 in Table A2) 

SI 

Onboard speaker (AC 1 ) 

0.000001 (assumed) 

S2 

Onboard speaker (AC2) 

0.000001 (assumed) 


Table A3: TSAFE components and failure probabilities 


D. TCAS and Pilot Visual Avoidance 

The TCAS safety layer shares two components with other AAC safety layers: 1) mode S transponder, and 2) 
cockpit speaker. If either of these components fails in a safety layer upstream or any other TCAS subcomponent 
fails (i.e., on-board TCAS computer, on-board radar, etc. 7 ' 13 ), then TCAS fails and there is a 30% chance that pilots 
will not be able to visually detect and avoid the NMAC. 6 In that case, there is a 28% chance of collision. 7 


Component 

Description 

Failure probability 

MSI 

Mode S: Resolution upload via data link (AC1) 

N/A (same as MSI above) 

MS2 

Mode S: Resolution upload via data link (AC2) 

N/A (same as MS2 above) 

SI 

Onboard speaker (AC 1 ) 

N/A (same as SI above) 

S2 

Onboard speaker (AC2) 

N/A (same as SI above) 

TCAS 1 other 

Non-transponder, non-speaker TCAS components (AC1) 

0.1002 (ref. [71, based on ref. [61, [181) 

TCAS2 0 ,her 

Non-transponder, non-speaker TCAS components (AC2) 

0.1002 (ref. [71, based on ref. [61, [181) 

VA 

Pilot detects and resolves the conflict via visual avoidance 

0.30 (ref. [61) 

CNMAC 

Collision given NMAC 

0.28 (ref. [71) 


Table A4: TCAS, pilot visual avoidance, and collision failure probabilities 
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